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Background 

During the past few years, several Harvard paper 
surveys were converted to Web surveys. These were 
high-profile surveys endorsed by the Provost and the 
Dean of the College, and covered major portions of the 
university population (all undergraduates, all graduate 
students, tenured and non-tenured faculty). When 
planning for these surveys started in 2001, Web surveys 
were still avant-garde. Dillman’s 2000 edition of his classic 
Mail and Internet Surveys: The Tailored Design Method 
devoted only 29 of 450 pages to Web surveys, indicative 
of the small size of the collected body of knowledge 
about Web surveys at that time. We did not have a good 
practical manual to guide us in the Web survey process, 
so we wrote our own. 

We have delivered versions of this paper at various 
conferences and at each venue, received a big group of 
professionals seeking ways to get started, or needing 
tips to refine their own processes. Since 2001 , our group 
has done several Web surveys together at Harvard, 
using a variety of software packages. We also have 
drawn upon the growing body of literature now available 
on Web surveys. 

In this paper, we will begin farther back in the process 
with how the survey projects were conceptualized and 
enacted in a complex university environment. We then 
take you through many organizational and technical 
aspects of doing a Web-survey. Along the way, we talk 
about some emerging issues surrounding survey 
research, such as privacy and incentives. 



Critical Background Questions 

The beginning of our project was similar to the 
beginning of all survey projects. There were four 
predominant reasons for conducting the surveys: 



Table 1 

Reasons to Do a Survey 



Top-Down 


University administration wants 
to know more about a 
population or subject area 
before formulating a 
recommendation. 


Bottom-Up 


University administration wants 
to make a policy or business- 
practice change, and needs 
data to support 
recommendation 


Consortial 

Project 


University participates in 
consortial project to have peer 
information. 


Trend Data 


University repeats previous 
survey to have trend data for 
decision-making. 
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For each project, the critical background questions 
were the same: 



Table 3 

Charge to Core Team Members 



Table 2 

Critical Background Questions 



1 . What is the purpose of the survey, and how will we use the 
survey data? 

2. Who are we surveying? 

3. Who are the audiences for the survey results? 

4. What kind of support do we need for the survey effort? 

* Building a survey team (see below) 

5. What survey methodology will we use? (see below) 

6. What is the project budget? 



The two survey projects that stemmed from the Harvard 
Office of Instructional Research and Evaluation were 
established consortial or trend-data surveys which already 
received support from the Dean of the College and had 
long-standing audiences for the survey results. However, 
a decision was made to switch both types of surveys 
from paper surveys to Web surveys. 

The two surveys initiated by Harvard Real Estate 
Services (HRES) had more complex origins 3 . The HRES 
staff traditionally conducted triennial surveys on housing 
and transportation topics. These surveys focused on 
different university populations with the aim to collect 
updated demographic information, information on current 
housing decisions and satisfaction, and information about 
the demand and need for Harvard housing and 
transportation services. An earlier incarnation of one of 
the surveys, the 2001 Harvard Graduate Student Housing 
Survey, was a paper survey. There was a need for trend 
data, more complete coverage of the topic (and thus, a 
redesigned survey), and a desire for a more streamlined 
administration via the Web. The Provost’s Advisory 
Committee on Housing recognized a need to understand 
faculty need and demand for housing and housing 
services, and the Committee recommended that a survey 
be undertaken in the spring of 2003 for the first time. 
Again, a decision was made to conduct the 2003 Faculty 
Housing and Transportation Survey via the Web. 

Building a Survey Team 

For the two original-design HRES survey projects, the 
members of the HRES Core Survey Team were charged 
with four tasks outlined in Table 3. 

To supplement HRES’s own expertise, several different 
types of consultants were retained to help with each 
survey (see Table 4). In addition, because the two HRES 
surveys were expected to have many and varied 
audiences for the results, a decision was made to establish 



1 . Identify areas of expertise needed to undertake the effort 
(both functional and topic specific) 



2. Identify if there are survey synergies and multiple topics that 
can be combined to prevent over-surveying the population 



3. Identify strengths and gaps across the internal resources 

4. Determine if budget allows for external professionals 



different internal survey teams and advisory groups to 
secure buy-in to the survey effort from various levels 
within the university. For example, the Advisory 
Committee for the Faculty Survey was explicitly 
established to help the survey effort in that each member 
could be a spokesperson within his or her department or 
school to other administrators and faculty. In addition, 
the establishment of a committee would help get the 
attention of the top decision-makers. The Advisory 
Committee was also helpful in reviewing wording that 
needed to be precise and appropriate for the respondent 
community (in particular, acronyms and building, place, 
and office nicknames). 

Harvard Real Estate Services Faculty Survey project 
ultimately included four teams: 



Table 4 

2003 Harvard Faculty Housing 
and Transportation Survey Team 



1 . Core Team 


The HRES staff; Office of 
Budgets, Financial Planning, 
and Institutional Research staff 


2. Outside Consultant Team 


Survey research specialist; 
architectural consultant; 
information technology 
consultant 


3. Internal Advisors 


Faculty members with area 
expertise or methodological 
expertise; Office of Instructional 
Research and Evaluation; 
Office of Budgets, Financial 
Planning and Institutional 
Research 


4. Advisory Committee 


Deans or directors 
representing each of Harvard's 
1 0 schools 
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Table 5 

Roles and Responsibilities of Survey Team Members 



1 . Executive sponsor 


Senior administrator (President/Provost, VP, Dean), often the person who signs 
the cover letters to participate in the survey 


2. Project director 


Head of the department/office charged with administering the survey 


3. Project manager 


Person with overall responsibility for the survey effort 


4. Content specialists 


Identify themes and topical areas, determine questions; liaise with Advisory 
Committee (see below) 


5. Survey research expert 


Overall knowledge of survey research, with special responsibility for appropriate 
question wording, survey design, and Web design. May be used to consult on all 
aspects of project including selection of appropriate survey population, and 
selection of Web programming or IT groups, or as a project manager. 


6. IT specialists 


Survey layout, HTML programming, manage other IT functions, e.g. host survey, 
create final database 


7. Keeper of e-mail list 


Knowledge of Human Resources or Registrar's databases, assembles e-mail list 
of survey population 


8. Communications 


Manage all communications between sponsoring administrator and target 
populations (e.g., launch letter, reminder letters, closing letters). 


9. (Sales Manager) 


(Note that some Web programming and IT groups follow a business model of 
organization by working through a sales manager who oversees many details but 
does not do the actual work. Your survey team may or may not have direct 
conversations with the actual programmer on your project.) 



General Roles and Responsibilities 
of Survey Team Members 

For any survey project, there are a number of survey 
functions that must be covered, and it is not uncommon 
for survey team members to have overlapping duties. 
With paper surveys, an institutional researcher might do 
all the work alone “from A to Z,” including pre-survey 
tasks such as copying the survey instrument, stuffing 
and stamping, and walking it to the Post Office. By 
contrast, Web surveys involve tasks that require 
specialized skills and it is more likely that there would be 
at least two people involved — a survey research specialist 
and an information technology specialist. 

Defining survey content. (Led by content specialist 
and survey researcher.) If the project involves an original 
survey, a large portion of the survey effort should be 
devoted to defining the survey content. Issues of policy- 
level importance to the university will likely inform the 
content. It is important to focus on the questions that 
your audiences need answers to in order to inform 



decision-making; questions should yield results that are 
useful, not simply interesting . In addition, if you have 
multiple audiences for your survey, each with their own 
agenda, take care to combine only related topics into a 
single survey at the risk of frustrating your respondent 
population. 

Some tips for defining survey content are in Table 6 on 
page 4. (Tips for programming Web survey questionnaires 
are found in Questionnaire Design.) 

Choosing Web Surveys over Paper Surveys 

Although this paper presumes that you made the 
decision to use a Web survey, it is helpful to be able to 
defend this decision. At some universities, students, staff 
and faculty expect campus communication to take place 
via the Web, and survey respondents appreciate the 
convenience of Web surveys. However, there are certain 
populations and scenarios where Web surveys may not 
be the best option. The advantages and disadvantages of 
web surveys are in Table 7 on page 4. 
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Table 6 

Tips on Defining Survey Content 



1 . Identify overarching themes that are of importance 
to the executive sponsor and other internal decision- 
makers. Seek early and timely confirmation/approval 
from the survey sponsor and other decision-makers. 

2. Use focus groups or interviews with members of 
target population to confirm themes and test 
sensitivities, if necessary. 

3. Write questions using the assistance of topical 
experts to be sure that questions will yield meaningful 
results that will inform policy setting, changes to 
business practice, etc. Avoid jargon, shorthand, and 
technical terms unless you are certain this language 
will make sense to everyone. 

4. "Paper test" questions with members of target 
audience to be sure that they (a) make sense to 
representatives of target audience and will be 
answered in the intended manner; (b) do not offend, if 
the topic is sensitive; (c) wording is appropriate for the 
times and the culture (e g., 'Black' vs. 'African- 
American'); (d) fixed-response questions include all 
the relevant options; (e) survey time does not exceed 
your goal length. (See Pre-Testing) 

5. Revise questions as appropriate. Whittle down 
questions to meet goals related to survey length. 

6. Provide questions with programming and design 
annotation (instructions about type of question, skip 
patterns, layout, etc.) to IT liaison for programming. 

7. Prepare the survey designer and the Web 
programmer from the very beginning that the survey 
might need a last-minute sign-off from a high-level 
administrator. Consequently, last minute changes may 
be involved, both big and small. 



Selection of Survey Population 

As anyone who has ever done government reporting, 
created a university factbook, or responded to a data 
request on population numbers knows, there is more 
than one way to define a population of students, faculty, 
alumni, parents, and staff. It is, of course, important to 
get the survey to the correct people, but it is also important 
to be able to document and, possibly, replicate the 
population selection in order to calculate the survey 
response rate. Often, an HR or IT person who is unfamiliar 
with the survey goals is the one who works with the 
administrative database to select the population and 
creates the e-mail list. Build the survey in a way to 
double-check the population selection; for example, you 
might summarize the background data of the population 



Table 7 

Advantages and Disadvantages of Web Surveys 



Advantages 


1. Savings in printing, postage, data entry. 


2. No data entry errors from hand-entry. (However, poor 
programming could lead to lost data.) 


3. Shortened timeframe to administer surveys (3 weeks with 
Web surveys, vs. 6 weeks or more with paper surveys). 


4. Easier and cleaner to provide skip patterns or survey 
sections customized to different respondent populations. 


5. Almost immediate access to data for analysis. 


6. Can easily link to background data, if appropriate (e.g., 
gender, grades, rank). 


Disadvantages 


7. Need programming and IT expertise. 


8. Certain populations are not comfortable with using personal 
computers (senior faculty). (Note: at many campuses, this is 
becoming less of a problem.) 


9 Must have accurate e-mail lists (e.g., a potential problem 
with students who use Hotmail or AOL accounts instead of 
school e-mail accounts, or with parents or alumni who have not 
provided their e-mail addresses). (Note: this is becoming less 
of a problem with many university populations, but could still be 
a problem in specific groups, for example, with part-time 
students.) 


1 0. Web surveys are not recommended for e-mail software 
that doesn't support Web access. Must be able to click on a 
URL provided in an e-mail and to have it bring respondent to a 
Web page. PINE is an example of unacceptable e-mail 
software. 


11 . Software must be appropriate for both PCs and Macs. 


12. Data provided via a Web survey are not anonymous, 
although the survey administrators may choose to keep the 
results confidential. 



and compare it to a factbook table, or search the database 
to make sure that people in a specific category are 
excluded. Here are five population selection pitfalls we 
encountered: 

1. Be careful to exclude the non-resident population if 
inappropriate for survey goals (e.g., overseas 
students, faculty on sabbatical). Example: if you are 
designing questions that get at current satisfaction 
with or use of local resources, stick to the “on 
campus” population. Note that a university e-mail 
address can mask the fact that the recipient is at a 
remote location, so it might be helpful to also screen 
by mailing address. 
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2. For a survey of parents, you must decide if you will 
survey divorced parents separately, or only one 
parent per student. 

3. Will the list of respondents change during the 
lifespan of the survey (e.g., any student enrolled in 
a class throughout the term, vs. all students enrolled 
in a class as of the opening day of class), or even 
from the time you select the population to the 
beginning of the survey? For example, if you 
intended to survey all students, might late-enrolling 
students be coming to you to complain that they 
were excluded from the survey? 

4. Select on the proper variable to define your 
population. In one project with which we are familiar, 
a university variable that was thought to distinguish 
Ph.D.s from other degree-seeking students was 



used, only to learn that this variable was not valid 
for one of the university schools. Thus, in that school, 
M.A. students were inadvertently surveyed along 
with Ph.D. students. 

5. It is important to provide a background dataset with 
IDs and demographic information on all recipients 
for response-rate analysis, analysis of 
representativeness of the respondent population, 
and any merging of survey responses with 
background data. Keep this file for reference. 

Designing a Web Survey 
Questionnaire Design 

The presiding rule of questionnaire design is to make 
it simple for the respondent to complete. The questionnaire 
should be “eye candy” — pleasing to look at, clear in 



Table 8 

Tips on Web Questionnaire Design 



1 . Keep survey length to something that can be done in 25 minutes or less. This is the ideal, though it is not always 
possible. 

2. Manage the respondents' expectations of survey length with a counter or progress bar which tells them where they are in 
the survey, e.g., Page 1 of 5, Page 5 of 5. 

3. No matter what the length, some respondents will not complete the survey. In our experience, this appears to be a 
problem of greater magnitude with Web surveys than for paper surveys. Be careful of the order of your sections; we do not 
recommend including crucial questions, such as demographic questions, or an overall evaluation question, in the final 
sections of the survey. (We search for ways to insert crucial questions closer to the middle of the survey because it is not 
usually appropriate to ask in the beginning.) 

4. The less clicking and typing for the respondent, the better. We have some evidence that respondents tend to skip over 
drop-down questions and write-ins for short-answer questions such as age or gender. Provide all options with radio buttons, 
in double columns, if necessary, if you don't want the question to take up too much room on the screen. 

5. Another reason to avoid write-ins for short answers is that respondents will rarely type their short answers the same way, 
which will involve a great deal of hand recoding at the analysis end. 

Example: Gender , vs. (better) Gender □ Male □ Female 

6. Although we prefer to display all options on the screen and not use drop-down boxes, there may be situations where you 
prefer to use a drop-down box for a long list of options. (For example, towns, states, occupations.) If you use a drop-down 
box, be sure that the default which appears on the screen is something like ''Click Here" and not a valid survey response 
such as "Boston," or "Strongly Disagree." You don't want the respondent's choice to be unnecessarily influenced by what 
they see on the screen. In addition, careless programming could turn a non-response into the default response. 

7. Use a delicate hand with color and only use color to help your respondent navigate the survey. For example, every other 
line in a long grid of questions can be shaded, and section headers or header and footers can be in a different color than 
the questions. Be aware that unusual colors might not appear the same to all end-users because of 1) the color palate of 
the end-user's computer, and/or 2) possible color-blindness. 

8. Restrain the inclination to use images. Graphics distract from the most important part of the questionnaire-the text. In 
addition, a file with images might load slowly. 

9. At least one general comment question at the end of the survey is useful-it provides good quotes to accompany the 
survey data analysis. In addition, it makes respondents feel that their opinions are valued. 

10. Each page should have a small distinctive logo/header on the top or bottom. A "help" e-mail or phone number should 
also be prominently displayed. Be careful, that the header/footer does not take up too much of the viewing page. 
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layout and meaning, flowing in structure, easy to complete. 
No incentive is high enough to induce respondents to 
waste their time on a confusing and frustrating 
questionnaire! There has been a lot written over the 
years on tips for paper questionnaire design, and many 
of these rules still apply for a Web questionnaire. 

However, designing a questionnaire for the Web has 
some additional and unique challenges. For one thing, 
everything is bigger and closer on the Web. Things that 
are out of line look particularly jarring, andfont and color 
clashes are right there to confront the respondent. A 
growing body of literature points to the variation in 
responses that result from different visual layouts for 
otherwise similar questions. Dillman, et al. (2005) writes 
about the use of multi-media effects in Web surveys, the 
lack of context in not seeing the entire survey or being 
able to flip backward and forward through the instrument, 
and the sophisticated possibilities for branching in Web 
surveys. (Also see Couper, M.P, et al., 2005.) We 



recommend familiarizing yourself with the new research 
literature and making strong use of pre-testing to see 
where you need to make questionnaire design adaptations 
for the Web context. 

In addition, the famous information design specialist 
Edward Tufte points out that, contrary to what one might 
think, a much smaller amount of information can be 
displayed on a Web screen compared to a piece of 
paper. Keep in mind that because of all the headers and 
footers provided by Windows applications, sometimes as 
little as one-third of the screen might have your 
questionnaire on it. 

Choosing HTML Software and Format 

We have been involved in Web projects that rely on 
many kinds of software to program the HTML, for example, 
Macromedia’s ColdFusion and Dreamweaver, 
SurveyMonkey, or Websurveyot®. Each program has its 
own complexities, and can be adequately customized if 



Table 9 

Thoughts about Software 



1 . Own programming vs. canned packages. Each canned package has some sort of limitation - some that 
will jeopardize the integrity of the responses, and some that will affect the graphic look of the design. 

Example 1) Once a question is answered, respondent can't erase responses to a question entirely, although 
he/she can change responses. To avoid this, provide a "not applicable" or some such response so that 
respondents can decide NOT to respond at all after initially giving an answer. 

Example 2) Might be initially difficult to evenly space a grid horizontally, or control hanging indents. 

2. Test the time it takes for each survey page to load (especially if using images); keep it short. Test using 
both a high-speed line and a dial-up line. 

3. Check to be sure survey works on both Macs and PCs. 



4. Determine which browsers are optimal, including which version of the browser, and give this info to 
respondent in the cover page (i.e., Netscape 8.0, Internet Explorer 6.0). Provide respondent with the link to 
download current software, if necessary. Optimally, the survey should work on all current popular browsers. 
As some respondents may be using older browser versions, it may be necessary to list the browser versions 
that are best in the cover page (e.g., 'Internet Explorer 6.0; Firefox 1 ,5.x.x') 

5. Output of data: 

• Data must be exportable to an SPSS datafile or other statistical software, Access, a comma or semi-colon 
delimited data file, or a .csv file. 

• Excel is not recommended because of size limitations for both the dataset and for individual text cells, but 
Excel is acceptable if there are a small number of respondents, the survey is short in length, and there are no 
lengthy comment fields. 

• Comments (longer essay questions) work well in Access (using memo fields), or Word, and should be 
associated with an ID field in order to link with background and other survey data. 
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the programmer is willing to take the time. Now is the 
time to start thinking about the important fact that a 
programmer is generally just a programmer, not a survey 
research and questionnaire design specialist or a graphic 
designer. No software will get you what you need if the 
survey researcher doesn’t take the time to work closely 
with the programmer. For example, you can’t assume 
that the programmer and survey researcher speak the 
same language with regards to coding, layout, or output 
format. More will be discussed on this in the next section. 

Technical Issues of Turning a Survey Questionnaire 
into a Web Survey 

If you are doing the programming yourself, start by 
reviewing the basics of questionnaire design. (Dillman’s 
book, Mail and Internet Surveys: The Tailored Design 
Method, is still the “bible.”) Then seek any help you might 
need on graphic design for the Web from a webmaster 
or s omeone i n a publications or g raphic d esign 
department. Think carefully about how you will analyze 
the questions at the end to help you program the skip 
patterns correctly. 

In the previous section, we mentioned that the 
programmer and survey researcher/data analyst don’t 
necessarily share an understanding of data technicalities. 
After all, the programmer’s job ends when the survey 
administration is completed; this is when the analyst’s 
job begins. As an example, in one project the survey 
researcher indicated very specifically that dichotomous 
variables were to be coded as 1=yes, and 2=no (wanting 
to avoid a situation where the programmer used 1=yes 
and a ssigned “ no” t o b lank). The p rogrammer u sed 
dichotomous codes, but relied on a programmer’s default 
of 1=yes and 0=no (and didn’t change the codebook!). 
This problem may have been avoided if the researcher 
had thought like a programmer as in our experience, it is 
more difficult for a programmer to think like a data analyst. 

In several projects, we have seen problems at the 
analytic stage with a question that involves a skip pattern 
followed by a “mark all that apply” type variable. 

If Q.2.a. or Q.2.b. weren’t marked, the programmer 
assigned their values as blank, and, thus, their responses 



Example: 

1. Do you have children living with you? 

□ Yes □ No (skip to Q. 3.) 

2. Please mark all the age categories in which 
you have children living with you? 

a. Infant to 4 years □ 

b. 5 to 9 years (etc.) □ 



could not be distinguished from those who were guided 
to skip the question; the analyst will need to do some 
post-survey programming to an alyze this correctly. 
However, another option is to change the question format 
of Q.2. from multiple-select to single-select. Now each 
person completing this section will have a response to 
each variable (no or yes), while persons skipping both 
Q.2.a. and Q.2.b. will be assigned to blantor missing for 
each variable, or, better yet, to a distinct code such as 
99. 



Example: 

1. Do you have children living with you? 

□ Yes □ No (skip to Q. 3.) 

2. Do you have any children living with you 
in any of these age categories? 

a. Infant to 4 years DNo □ Yes 

b. 5 to 9 years (etc.) DNo □ Yes 



Here are some general technical issues to consider: 

1 . You must decide if the respondents will be required 
to respond to any or all of the questions. Requiring 
anyone to respond to the survey as a whole as well 
as individual questions often does not fit with the 
university’s ethos. It also might conflict with 
instructions you receive from your Human Subjects 
Review Committee. (See Privacy Issues, page 8) 

In addition, with certain populations, it could simply 
annoy the respondents. However, a required 
response might be necessary for proper coding of 
skip patterns. 

2. Create the survey as multiple pages with a submit 
button at the end of each section, and not as one 
long page. This helps the respondent easily navigate 
through the survey. In addition, each individual page 
can be submitted (and captured) when completed, 
which will prevent the loss of data if the respondent 
doesn’t finish the entire survey or if the system 
crashes. 

3. You must decide if respondents will be able to exit 
and re-enter the survey, and if yes, whether or not 
they will be able to change any or all of their 
responses. If the survey is created as multiple pages 
(as described above), one method might be to 
allow the respondents to re-enter the survey at the 
last unfinished page, and allow them to enter and 
change their responses going forward (but not the 
pages already submitted). Be sure the respondents 
know whether they can exit and re-enter the survey. 
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4. Pre-determine the target population’s technical 
knowledge about the survey subject. You might want 
to consider providing a pop-up glossary of terms 
that opens in a new window. 

Example: “What percentage of your monthly after- 
tax budget do you spend on your housing?” [Pop- 
up definition: “Monthly After-Tax Budget: all monies 
available to you to spend on housing, food, books, 
insurance, social activities, health, clothing, etc.”] 

5. Select a graphic design template that is emblematic 
of survey sponsor’s image or the image of the college 
or university. 

• Choose a distinctive color scheme and appropriate 
fonts for headers and question text. Get help 
from a professional g^aphic designer, if necessary. 

• Consider shading alternate rows in big grids of 
questions. 

• Web pages should have margins on all sides to 
make the page readable. 

6. Take time to be sure the programmer understands 
the questionnaire in order to program skip patterns 
correctly. In addition, make sure the programmer 
understands the different types of questions. (See 
codebook example, Table 10.) 

7. Create the survey as multiple pages, and not as one 
long page. This helps the respondent navigate 
through the survey more easily. In addition, each 
individual page can be submitted (and captured) 
when completed, which will prevent the loss of data 
if the respondent doesn’t finish the entire survey if 
the system crashes. 

8. The survey designer should provide a codebook or 
data dictionary of survey questions and possible 
responses for programmer to work with. (See Table 
10 .) 

9. Obtain or create a print copy of the final Web survey 
for inclusion in a survey report and for historical 
record. It may be especially important to get a paper 
copy if the survey resides on a server that is not 
under the control of the survey group (e.g., another 
university office, an outside consultant). 

Note that Web software packages generally do not 
provide a pretty print copy — what looks fine on the Web 
may not print well in hard copy. Although time-consuming, 
we find that it works well to copy each question, one at a 
time, from the Web browser, and paste it into a blank 
Word document with your desired header and footer. 
Then adjust the formatting accordingly — line and column 
widths, borders, disabling all macros that are pasted in, 
and deleting extraneous characters that were created by 
macro functions. 



Here is an example of a fully-documented codebook. 

Table 10 

Example of Codebook, and Different Types 
of Survey Questions 



Q1 . W ho o wns y our h ousing? ( Single select 
question. Option codes: 1,2, blank . ) 

O Harvard owned 
O Privately owned ( Skip to Q4.) 

What resources did you use to find your current 
housing? (Check all that apply.) ( Multiple select 
question. For each response, option codes 0, 1, 99 
(intentionally skipped)) 

Q2a. □ Residence Halls application ( Skip to Q5 . ) 
Q2b. □ Harvard Housing Office ( Skip to Q5 . ) 

Q2c. □ Other (Continue with Q3.) 

Q3. If you selected ‘Other,’ please specify: ( Text 
response question . ) 



Administering the Web Survey 

Technical Issues of Administering the Web Survey 

There are many small, but important, decisions to be 
made about how to administer the survey. It is important 
that these decisions be tailored to the culture and style 
of your university, or to the standards of your office. 
Remember that your ethos should guide your systems; 
don’t let your system defaults force you to change your 
standards! 

Pre-Testing 

Pre-testing takes time, but it is an important step. It’s 
akin to always asking for references — even if you have 
never had any problems, you know that it’s inevitable 
that you will some day if you don’t ask for them. Similarly, 
you should always have someone from the population 
you are surveying look at the questionnaire with fresh 
eyes and provide comments. 

Privacy Issues 

There are two sides to the privacy issues. The biggest 
privacy issue is vis a vis the respondents. Every school 
has a Human Subjects Review Committee/lnstitutional 
Review Board (IRB) set up to review all types of projects 
involving human subjects. Human Subjects Review is 
very prominent for projects and experiments conducted 
by a Medical School or department of Psychology. In our 
experience, many schools, exempt administrative surveys 
from Human Subjects Review. It is important to determine 
the position of your specific IRB. We found the Harvard 
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Table 11 

Administering the Web Survey 



1 . What is the desired timeline for the survey? We have found that 3 weeks is about right, with e-mails sent 4 days to 1 week apart. We adjust 
the timeline based upon the response rate. If the response is flagging, we might send a reminder e-mail one day earlier than originally 
scheduled. 

2. How many contacts? We have been contacting the respondent 3 to 4 times: (if needed as justification for the survey) a pre-letter from a top 
administrator (President/Provost, VP, Dean); a launch letter with the .survey URL; a follow-up e-mail only to those who have not responded which 
also announces the end of the survey a few days hence; a thank-you letter that goes to all respondents. 

3. Detail on suggested e-mail contacts: 

• Optional pre-survey notification letter from top administrator to notify the population that the survey is coming and why it is important to the 
university. 

• Launch letter/email containing the survey link and contact e-mail and contact name in the event that a recipient has questions/comments. 

• Pre-assigned unique URL allows survey team to send 1 or 2 follow-ups to only non-respondents Sending reminder letters to people who 
completed the survey will only cause frustration, confusion, and a slew of e-mails to project manager. The logon process can be used to select 
those who need reminders. 

• 2 nd follow-up could also be a notification of closing letter: "The survey will be closing on May 6th at 5.00 p.m." 

• Thank you e-mail goes to all respondents. Include information on response rate or something learned from the survey, if desired. 

• Place "survey is closed" page at the survey address for latecomers to see. 

5. Choose an appropriate subject line for all e-mail communications. We have used: 

• A message from Provost Smith 

• A reminder from Provost Smith 

Be aware that some subject lines with valid meanings may be identified by certain browsers as junk mail, e.g., 'Congratulations,' or 
'Confidential.' 

6. How will respondents enter into the survey? We have used two methods: a login via PIN authentication, and unique URLs for each 
respondent. 

7. Proper programming is needed to ensure that one person submits only one survey. 

8. What constitutes a completed survey? 

• Suggestion: in a multi-page formatted survey, after completing each page, have a click button that submits this page. (Could have 2 buttons, 
'Save and logout' and 'Continue to next page') If re-entering the survey, enter at beginning of first uncompleted page (entry on that page not 
saved). 

• Who should receive reminders to complete the survey? We suggest all respondents who have not submitted the final page of the survey. 

• Output data can capture all records, including those with only partial data, or only completed records. We prefer to capture output data from all 
who have entered the survey. Although some records will be incomplete, we believe they contain valuable responses to a portion of the 
questions. 

9. How are respondents required to navigate the survey? We use the fixed order of pages method - respondents can jump around on a 
particular page, but must do page 1 and submit it before going to page 2. 

10. Time of day to send e-mails: 

• Evening often works well when respondents are done with the business of the day and ready to relax and look at non-work e-mails. 

• If IT folks send the e-mails, it will depend on their schedules and the acceptable number of e-mails they can send at one time. The server must 
have the resources to temporarily store all of the emails until the respondents download them, and to do both the task of sending out these 
emails and other server chores. If not, it may be necessary to send out the survey in small batches. 

1 1 . Day of week to send e-mails: 

• Differs from survey wisdom learned from paper surveys. 

• It is acceptable to have Web surveys arrive at any day of the week, including weekends. 

12. It is important to monitor the response rate while the survey is live. Use the time-date stamp that comes with each response to analyze the 
data, for example, responses by School and Faculty Rank. This should be used to: 

• help target dates for reminders 

• change or customize the reminder text 

• catalog for future projects the days and times that your survey population prefers to respond to surveys 

• manage the expectations of top administrators as to how many days it will take to administer the survey and the eventual anticipated response 
rate. 
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Table 12 
Pre-Testing 

1 . Paper testing. When the survey is still in the paper-stage, ask individuals or assemble a group of 
respondents to look for terms that they do not understand, confusing question wording, and obvious questions 
omitted from the survey. Be attentive to their comments, but remember that they are only a small group of 
people; your overall sense of the survey structure should predominate. 

2. Before the Web survey is finalized, have individuals take the survey using a variety of different computer 
systems, e.g., Macs, PCs, dial-up and high-speed connections, and with different browsers. All should test 
whether the survey flows properly, whether there are any problems with navigating from page to page, with 
pop-up glossary boxes, or with incorrect skip patterns. 

3. Save the responses from the Web testers and create a test output dataset to test the integrity of the 
responses and if there is a need for any custom programming (e.g., variable names or value labels that are 
only assigned at the output state, custom value codes assigned to skipped items). Run frequencies to make 
sure that the output coding is as requested. Be sure to exclude any pre-test response cases from your final 
operational dataset. 

4. Remind the Web programmer regularly that there may be some last minute changes based on feedback 
from pre-testing. 



IRB very helpful for determining the appropriate language 
to be used in e-mail communications. 

The second privacy issue is forconsultants you might 
use. It is important to have a clear agreement regarding 
the capture, storage, and destruction of the survey data 
with any survey research or Web-programming 
consultants. Some schools also require survey research 
consultants to have Human Subjects Review Certification. 
(This can be done over the Web at the National Institute 
of Health Web site 4 .) 

It is also important to clarify Web-programming 
company policies on whether or not they append 
advertising for their product to your survey. 

For privacy purposes, all survey data should be stored 
on a secure server. 



Survey Incentives 

With a paper survey, the administrator can include an 
actual object in the mailing. Fairly recently, in the 
commercial world, the “going rate” was still $1 or $5. 
Dillman always stressed that pre-payment was more 
powerful than post-payment (after completing the survey). 
However, we are not aware of pre-payment as a norm for 
university surveys. 

A number of schools we are familiar with received 
good boosts in survey response from offering creative 
and desirable gift incentives. A clever way to combine an 
incentive to complete the survey with an incentive to 
respond quickly is to have those who complete the survey 
early entered multiple times into a lottery. The introduction 
could say something like: 



Table 13 

Privacy Issues: A Harvard Example * 



1. Survey was deemed exempt by the Harvard Institutional Review Board (IRB) because the survey did not 
meet the Federal definition of research; the survey was not "a systematic investigation designed to contribute 
to 'generalizable' knowledge," as the results were being used for in-house planning. 

2. The Harvard attorneys thought that it was unnecessary to include consent language for the recipients; login 
was considered consent. 



3. The attorneys also recommended including language to assure respondents that their responses would 
not be personally identified with them other than to the survey research analyst, (i.e., anonymous). 

4. Results will be aggregated for the University report - no individual data will be released, (i.e., confidential). 



5. Any comments made by respondents that include either the name of the respondent or the names of 
university personnel should have the names removed before being circulated. 

* Seek specific advice from your university attorneys or IRB. 
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Complete the survey by Tuesday and be entered into 
the gift lottery 1 0 times! 

Complete the survey by Thursday and be entered into 
the gift lottery 5 times! 

This technique has been used with student surveys 
and alumni surveys. It does not seem particularly 
appropriate for a survey of the faculty or parent population. 
Let the culture of your school be your guide as to when 
incentives are and are not appropriate. 

One caveat: One research study on incentive usage 
showed that although there was a significant difference 
in response rates when an incentive was offered, in fact, 
the response rate difference was minimal and was a 
small impact on the substantive findings. This brings up 
the question as to whether time and money should be 
spent on awarding prizes, or on other efforts to improve 
response rates. (Porter and Whitcomb, 2004.) (Note: this 
research study used a survey in which participants had 
little personal reason in which to participate; findings may 
be different for a more cogent popular topic.) 

Table 14 
Incentives 



1. Be mindful of your school's policies on gifts and imputed 
income. At Harvard, cash gifts are not allowed and gift 
certificates are discouraged for students and not allowed for 
employees. Check if there is an upper limit on the value of a 
gift beyond which it is considered taxable income, as this 
could be a problem for students. There may also be 
separate rules regarding gifts to students who are on the 
school's payroll. 

2. Small gifts should be considered a thank-you, but not truly 
an incentive (e.g., a coupon for free coffee and muffin). 

3. Use a lottery to draw prizes. 



4. Make sure your gift is something that will attract your 
respondents. In 2005, for example, a portable CD player had 
little draw value while an iPOD had greater value. Next year, 
it is likely that there will be a new and different electronic 
device that will be the obvious and appropriate incentive. 

5. Two suggested ways to offer gift incentives: 

* Offer a significant number of opportunities to win good-value 
gifts (e.g., 40 gifts valued at $20-$40 each). 

* Offer one large grand prize such as a travel certificate, a 
few moderate-priced gifts (university chairs are nice for 
parents or alumni), and a market basket of very small gifts. 

6. Mail gifts to recipients, if possible. Requiring the gifts to 
be picked up can take a surprisingly large amount of 
coordination and staff time. 



Communication and Technical Support 

Hopefully, if something goes wrong with the survey 
administration, it will be something technical that you can 
control and correct. 

1. Here is a short laundry list of things that could go 
wrong: 

• the links don’t work 

• the recipient does not know how to download 
needed software 

• there are server problems when sending out too 
many bulk e-mails at once; (possible solution: 
send e-mails in waves) 

• the server could crash while the respondent is 
logged in 

• the respondent’s computer doesn’t have enough 
memory to load survey 

• the survey looks out of alignment on respondent’s 
computer. 

• the large numbers of e-mails sent to non-university 
e-mail addresses could be blocked by the 
recipients’ internet service providers; (same 
solution as above: send e-mails out in waves) 

2. The survey instrument should prominently display 
the phone number and/or e-mail address of a 
technical contact person for respondents who 
encounter pro blems with Web administration. 
Respond promptly to every inquiry. Rule of thumb: 
respond to every query within 1 business day, if not 
sooner. The technical support person should 
document the problems and summarize for the 
project manager. 

3. Some respondents will write personal notes about 
the survey content to the technical contact. These 
comments should be forwarded to the project 
manager to be read. 

4. Surveys sent to incorrect or outdated e-mail 
addresses will be returned. The technical support 
person should catalog this because it affects the 
calculation of the response rate. 

Output of Survey Data 

If the programming was done correctly (skips and 
assignment of missing values), this step is almost a non- 
step. One of the great advantages of Web surveys is 
immediate access to a final (or even interim) dataset. In 
Section 11.8. we talked about the importance of the survey 
researcher and programmer speaking the same language. 
Here’s another example: be sure to explicitly tell the 
programmer that you want data values output as numeric 
and not as text (i.e., ‘1-2-3,’ and not ‘Always - Sometimes 
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- Never’); there is a choice of output format in many 
programs. 

Reportage 

Don’t let anyone tell you that all they need are the 
frequencies and that no report is necessary! The 
frequencies and means are still just data, and not yet 
useful information. Here are some suggestions for ways 
to develop your analysis and make your project results 
useful now and in the future. 

Table 15 

Questions about Survey Analysis, and Reportage 



1 . Create a full set of frequencies and crosstabs as a paper 
appendix for posterity's sake. Paper is still more portable than 
a datafile between offices and overtime. 

2. A dataset is like a fingerprint--no two analysts will label, 
recode, and refine the dataset in the same way. Keep (and 
share!) documentation, e.g., calculating a mean of hours per 
week, where the top category, 20 +, is recoded as 22/hours 
per week. 

3. A decision should be made as to whether other university 
personnel will have access to the dataset to pursue analyses 
related to the project, or even unrelated goals. If you have 
developed recodes or created new variables crucial to the 
analysis, be sure to document this so that your analyses aren't 
questioned when others cannot duplicate them. 

4. It is true that a picture is worth 1 ,000 words. Take the time 
to become an expert designer of charts and tables. 

5. All your survey results will appeal to your direct clients, those 
who initiated the project. Small parts of the survey will appeal 
to other university offices that may or may not be one of the 
original intended audiences. Be creative about who might be 
interested in which pieces of data. 

6. Some of the audiences for the Harvard Faculty Survey were: 

• Harvard Real Estate Services 

• President's Office 

• Vice Presidents of Administration and Finance 

• Office of the Provost 

• Deans of Faculty Affairs and Student Life (for each school) 

• Harvard Mortgage and Educational Loan Office 

• Harvard Transportation and Commuter Services 



End Note 

Dillman describes a successful survey in the context 
of s ocial e xchange t heory a s a t rusting r elationship 
between surveyor and respondent where the ultimate 
rewards of the survey outweigh the costs of responding 
(Dillman, 2000, p.14-24). Through excellent survey design 
and a smooth (Web) administration, we act to reduce the 
survey cost to the respondents by making it easy and, 
hopefully, interesting, for them to participate. In each 



project, we received at least one response that says, 
“thank you for asking.” We learned that a positive 
encounter with respondents from the school community 
is, in itself, a desirable outcome of the survey project, in 
addition to providing information needed for decision- 
making. 

Endnotes 

1 Formerly of Harvard Planning and Allston Initiative 

2 Formerly of Harvard Office of Instructional 
Research and Evaluation, Faculty of Arts and 
Sciences 

3 For a review of many of the methodological issues 
in survey design that were considered with one of 
the Harvard surveys, as well as a copy of the 2001 
Harvard Graduate Student Survey on Housing and 
Transportation, see Harvard Business School Case 
9-505-059: Wathieu, Luc, The Harvard Graduate 
Student Housing Survey, (http:// 
www.hbsp.Harvard.edu) 

4 As of November 2006, http://206.102.88.10/ 
nihtrainina/ohsrsite/cbt/cbt.html . 

References 

Couper, M.P., Conrad, F.G., &Tourangeau, R. (2005). 
Visual Context Effects in Web Surveys, The American 
Association for Public Opinion Research (AAPOR) 60 th 
Annual Conference. 

Dillman, D.A. (2000). Mail and Internet Surveys: The 
Tailored Design Method, second edition. John Wiley and 
Sons, New York. 

Dillman, D.A., & Christian, L. M. (2005). Survey mode 
as a source of instability in responses across surveys,” 
Field Methods, 17, 1, pp. 30-52. SAGE Journals Online 
( http://online.sagepub.com/ ) 

Porter, S., & Whitcomb, M. (2004). Understanding 
the effect of prizes on response rates, overcoming survey 
research problems , New Directions for Institutional 
Research, No. 121 , pp. 51-62. 

Porter, S. (2004). Pros and cons of paper and electronic 
surveys, overcoming survey research problems, New 
Directions for Institutional Research, No. 121, pp. 91-99. 

T ufte, E. (1983). The Visual Display of Quantitative 
Information. Graphics Press, Cheshire, Connecticut. 

Wathieu, L. (2005). The Harvard Graduate Student 
Housing Survey. Harvard Business School Case Study 
9-505-059, Cambridge, Massachusetts ( http:// 
www.hbsp.Harvard.edu ). 

For questions oradditional information, please contact 
Rena Cheskis-Gold, www.demoaraphicperspectives.com . 
(203) 397-1612. 



14 



AIR Professional File, Number 102, Essential Steps for Web Surveys ... 



THE AIR PROFESSIONAL FILE— 1978-2006 

A list of titles for the issues printed to date follows. Most issues are “out of print,” but microfiche or photocopies are available through 
ERIC. Photocopies are also available from the AIR Executive Office, 222 Stone Building, Florida State University, Tallahassee, FL 
32306-4462, $3.00 each, prepaid, which covers the costs of postage and handling. Please do not contact the editor for reprints of 
previously published Professional File issues. 

Organizing for Institutional Research (J. W. Ridge; 6 pp; No. 1) 

Dealing with Information Systems: The Institutional Researcher's Problems and Prospects (L.E. Saunders; 4 pp; No. 2) 

Formula Budgeting and the Financing of Public Higher Education: Panacea or Nemesis for the 1980s? (F.M. Gross; 

6 pp; No. 3) 

Methodology and Limitations of Ohio Enrollment Projections (G.A. Kraetsch; 8 pp; No. 4) 

Conducting Data Exchange Programs (A.M. Bloom & J.A. Montgomery; 4 pp; No. 5) 

Choosing a Computer Language for Institutional Research (D. Strenglein; 4 pp; No. 6) 

Cost Studies in Higher Education (S.R. Hample ; 4 pp; No. 1) 

Institutional Research and External Agency Reporting Responsibility (G. Davis; 4 pp; No. 8) 

Coping with Curricular Change in Academe (G.S. Melchiori; 4 pp; No. 9) 

Computing and Office Automation — Changing Variables (E.M. Staman; 6 pp; No. 10) 

Resource Allocation in UK. Universities (B.J.R. Taylor; 8 pp; No. 11) 

Career Development in Institutional Research (M.D. Johnson; 5 pp; No 12) 

The Institutional Research Director: Professional Development and Career Path (W.P. Fenstemacher; 6pp; No. 13) 

A Methodological Approach to Selective Cutbacks (C.A. Belanger & L. Tremblay; 7 pp; No. 14) 

Effective Use of Models in the Decision Process: Theory Grounded in Three Case Studies (M. Mayo & R.E. Kallio; 8 pp; 

No. 15) 

Triage and the Art of Institutional Research (D.M. Norris; 6 pp; No. 16) 

The Use of Computational Diagrams and Nomograms in Higher Education (R.K. Brandenburg & W.A. Simpson ; 8 pp; No. 1 7) 

Decision Support Systems for Academic Administration (L.J. Moore & A.G. Greenwood; 9 pp; No. 18) 

The Cost Basis for Resource Allocation for Sandwich Courses (B.J.R. Taylor; 7 pp; No. 19) 

Assessing Faculty Salary Equity (C.A. Allard; 7 pp; No. 20) 

Effective Writing: Go Tell It on the Mountain (C.W. Ruggiero, C.F. Elton, C.J. Mullins & J.G. Smoot; 7 pp; No. 21) 

Preparing for Self-Study (F. C. Johnson & M.E. Christa! ; 7 pp; No. 22) 

Concepts of Costand Cost Analysis for Higher Education (P.T. Brinkman & R.H. Allen; 8 pp; No. 23) 

The Calculation and Presentation of Management Information from Comparative Budget Analysis (B.J. R. Taylor; 1 0 pp; No. 24) 

The Anatomy of an Academic Program Review (R.L. Harpel; 6 pp; No. 25) 

The Role of Program Review in Strategic Planning (R.J. Barak; 7 pp; No. 26) 

The Adult Learner: Four Aspects (Ed. J.A. Lucas; 7 pp; No. 27) 

Building a Student Flow Model (W.A. Simpson; 7 pp; No. 28) 

Evaluating Remedial Education Programs (T. H. Bers; 8 pp; No. 29) 

Developing a Faculty Information System at Carnegie Mellon University (D.L. Gibson & C. Golden; 7 pp; No. 30) 

Designing an Information Center: An Analysis of Markets and Delivery Systems (R. Matross; 7 pp; No. 31) 

Linking Learning Style Theory with Retention Research: The TRAILS Project (D.H. Kalsbeek; 7 pp; No. 32) 

Data Integrity: Why Aren’t the Data Accurate? (F.J. Gose; 7 pp; No. 33) 

Electronic Mail and Networks: New Tools for Institutional Research and University Planning (D.A. Updegrove, J.A. Muffo & J.A. 

Dunn, Jr.; 7pp; No. 34) 

Case Studies as a Supplement to Cuantitative Research: Evaluation of an Intervention Program for High Risk Students (M. Peglow- 
Hoch & R.D. Walleri; 8 pp; No. 35) 

Interpreting and Presenting Data to Management (C.A. Clagett; 5 pp; No. 36) 

The Role of Institutional Research in Implementing Institutional Effectiveness or Outcomes Assessment (J.O. Nichols; 6 pp; No. 37) 
Phenomenological Inten/iewing in the Conduct of Institutional Research: An Argument and an Illustration (L. C. Attinasi, Jr.; 8p p; No. 
38) 

Beginning to Understand Why Older Students Drop Out of College (C. Farabaugh-Dorkins; 12 pp; No. 39) 

A Responsive High School Feedback System (P.B. Du by; 8 pp; No. 40) 

Listening to Your Alumni: One Way to Assess Academic Outcomes (J. Pettit; 1 2 pp; No. 41) 

Accountability in Continuing Education Measuring Noncredit Student Outcomes (C.A. Clagett & D.D. McConochie; 6pp; No. 42) 

Focus Group Interviews: Applications for Institutional Research (D.L. Brodigan; 6 pp; No. 43) 

An Interactive Model for Studying Student Retention (R.H. Glover & J. Wilcox; 12 pp; No. 44) 

Increasing Admitted Student Yield Using a Political Targeting Model and Discriminant Analysis: An Institutional Research Admiss ions 
Partnership (R.F. Urban; 6 pp; No. 45) 

Using Total Quality to Better Manage an Institutional Research Office (M.A. Heverly; 6 pp; No. 46) 

Critique of a Method For Surveying Employers (T. Santa, R.H. Phillippi & W. Lyons; 8 pp; No. 47) 

Plan-Do-Check-Act and the Management of Institutional Research (G.W. McLaughlin & J.K. Snyder; lOpp; No. 48) 

Strategic Planning and Organizational Change: Implications for Institutional Researchers (K.A. Corak&D.P. Wharton; 10pp;No. 49) 
Academic and Librarian Faculty: Birds of a Different Feather in Compensation Policy? (M.E. Zeglen & E.J. Schmidt; 10 pp; No. 50) 
Setting Up a Key Success Index Report: A How-To Manual (M.M. Sapp; 8 pp; No. 51) 

Involving Faculty in the Assessment of General Education: A Case Study (D. G. Underwood & R. H. Nowaczyk; 6 pp; No. 52) 





AIR Professional File, Number 102, Essential Steps for Web Surveys.... 



15 



THE AIR PROFESSIONAL FILE— 1978-2006 

Using a Total Quality Management Team to Improve Student Information Publications (J.L. Frost & G.L. Beach; 8 pp; No. 53) 

Evaluating the College Mission through Assessing Institutional Outcomes (C.J. Myers & P.J. Silvers; 9 pp; No. 54) 

Community College Students’ Persistence and Goal Attainment: A Five-year Longitudinal Study (K.A. Conklin; 9 pp; No. 55) 

What Does an Academic Department Chairperson Need to Know Anyway? (M.K. Kinnick; 1 1 pp; No. 56) 

Cost of Living and Taxation Adjustments in Salary Comparisons (M.E. Zeglen & G. Tesfagiorgis; 14 pp; No. 57) 

The Virtual Office: An Organizational Paradigm for Institutional Research in the 90’s (R. Matross; 8 pp; No. 58) 

Student Satisfaction Surveys: Measurement and Utilization Issues (L. Sanders & S. Chan; 9 pp; No. 59) 

The Error Of Our Ways; Using TQM Tactics to Combat Institutional Issues Research Bloopers (M.E. Zeglin; 1 8 pp; No. 60) 

How Enrollment Ends; Analyzing the Correlates of Student Graduation, Transfer, and Dropout with a Competing Risks Model (S.L. 
Ronco; 14 pp; No. 61) 

Setting a Census Date to Optimize Enrollment, Retention, and Tuition Revenue Projects (V. Borden, K. Burton, S. Keucher, F. 
Vossburg-Conaway; 12 pp; No. 62) 

Alternative Methods For Validating Admissions and Course Placement Criteria (J. Noble & R. Sawyer; 12 pp; No. 63) 

Admissions Standards for Undergraduate Transfer Students: A Policy Analysis (J. Saupe & S. Long; 1 2 pp; No. 64) 

IR for IR-Indispensable Resources for Institutional Researchers: An Analysis of AIR Publications Topics Since 1 974 (J. Volkwei n & V. 
Volkwein; 12 pp; No. 65) 

Progress Made on a Plan to Integrate Planning, Budgeting, Assessment and Quality Principles to Achieve Institutional Improvemen t 
(S. Griffith, S. Day, J. Scott, R. Smallwood; 12 pp; No. 66) 

The Local Economic Impact of Higher Education: An Overview of Methods and Practice (K. Stokes & P. Coomes; 16 pp; No. 67) 
Developmental Education Outcomes at Minnesota Community Colleges (C. Schoeneck er, J. Evens &L.B oilman: 1 6 pp; No. 68) 

Studying Faculty Flows Using an Interactive Spreadsheet Model (W. Kelly; 16 pp; No. 69) 

Using the National Datasets for Faculty Studies (J. Milam; 20 pp; No. 70) 

Tracking Institutional leavers: An Application (S. DesJardins, H. Pontiff; 14 pp; No. 71) 

Predicting Freshman Success Based on High School Record and Other Measures (D. Eno, G. W. McLaughlin, P. Sheldon & P. 
Brozovsky; 12 pp; No. 72) 

A New Focus for Institutional Researchers: Developing and Using a Student Decision Support System (J. Frost, M. Wang & M. 
Dalrymple ; 12 pp; No. 73) 

The Role of Academic Process in Student Achievement: An Application of Structural Equations Modeling and Cluster Analysis to 
Community College Longitudinal Data 1 (K. Boughan, 21 pp; No. 74) 

A Collaborative Role for Industry Assessing Student Learning (F. McMartin ; 12 pp; No. 75) 

Efficiency and Effectiveness in Graduate Education: A Case Analysis (M. Kehrhahn, N. L. Travers & B. G. Sheckley; No. 76) 

ABCs of Higher Education-Getting Back to the Basics: An Activity-Based Costing Approach to Planning and Financial Decision 
Making (K. S. Cox, L. G. Smith & R.G. Downey; 12 pp; No. 77) 

Using Predictive Modeling to Target Student Recruitment: Theory and Practice (E. Thomas, G. Reznik &W. Dawes; 12pp;No. 78) 

Assessing the Impact of Curricular and Instmctional Reform - A Model for Examining Gateway Courses 1 (S.J. Andrade; 1 6 pp; No. 79) 
Surviving and Benefitting from an Institutional Research Program Review (W.E. Knight; 7 pp; No. 80) 

A Comment on Interpreting Odds-Ratios when Logistic Regression Coefficients are Negative (S.L. DesJardins; 7 pp; No. 81) 

Including Transfer-Out Behavior in Retention Models: Using NSC EnrollmentSearch Data (S.R. Porter; 16 pp; No. 82) 

Assessing the Performance of Public Research Universities Using NSF/NCES Data and Data Envelopment Analysis Technique (H. 
Zheng & A. Stewart; 24 pp; No. 83) 

Finding the Start Line’ with an Institutional Effectiveness Inventory 1 (S. Ronco & S. Brown; 12pp; No. 84) 

Toward a Comprehensive Model of Influences Upon Time to Bachelor’s Degree Attainment (W. Knight; 18 pp; No. 85) 

Using Logistic Regression to Guide Enrollment Management at a Public Regional University (D. Berge & D. Hendel; 14 pp; No. 86) 

A Micro Economic Model to Assess the Economic Impac t of Universities: A Case Example (R. Parsons & A. Griffiths; 24 pp; No . 87) 
Methodology for Developing an Institutional Data Warehouse (D. Wierschem, R. McBroom & J. McMillen; 12 pp; No. 88) 

The Role of Institutional Research in Space Planning (C.E. Watt, B.A. Johnston. R.E. Chrestman & T.B. Higerd; 10 pp; No. 89) 

What Works Best? Collecting Alumni Data with Multiple Technologies (S. R. Porter & P.D. Umback; 10 pp; No. 90) 

Caveat Emptor: Is There a Relationship between Part-Time Faculty Utilization and Student Learning Outcomes and Retention? 

(T. Schibik & C. Harrington; 10 pp; No. 91) 

Ridge Regression as an Alternative to Ordinary Least Squares: Improving Prediction Accuracy and the Interpretation of Beta Weig hts 
(D. A. Walker; 12 pp; No. 92) 

Cross-Validation of Persistence Models for Incoming Freshmen (M. T. Harmston; 14 pp; No. 93) 

Tracking Community College Transfers Using National Student Clearinghouse Data (R.M. Romano and M. Wisniewski; 14 pp; No. 94) 
Assessing Students’ Perceptions of Campus Community: A Focus Group Approach (D.X. Cheng; 1 1 pp; No. 95) 

Expanding Students’ Voice in Assessment through Senior Survey Research (A.M. Delaney; 20 pp; No. 96) 

Making Measurement Meaningful (Carpenter-Hubin, J. & Hornsby, E.E., 14 pp; No. 97) 

Strategies and Tools Used to Collect and Report Strategic Plan Data (Blanked, J., Lucas, C. & Frost, J.; 14 pp; No. 98) 

Factors Related to Persistence of Freshmen, Freshman Transfers, and Non fresh man Transfer Students (Perkhounkova, Y, Noble, J & 
McLaughlin, G.; 12 pp; No. 99) 

Does it Matter Who’s in the Classroom? Effect of I nstructor Type on Student Retention, Achievement and Satisfaction (Ronco, S. & 
Cahill, J.; 16 pp; No. 100) 

Weighting Omissions and Best Practices When Using Large-Scale Data in Educational Research (Hahs-Vaughn, D.L.; 12 pp; 
No. 101) 





AIR Professional File, Number 102 Essential Steps for Web Surveys.... 



16 



The AIR Professional File is intended as a presentation of papers which synthesize and interpret issues, 
operations, and research of interest in the field of institutional research. Authors are responsible for material 
presented. The File is published by the Association for Institutional Research. 



Editor: 

Dr. Gerald W. McLaughlin 
Director of Planning and 
Institutional Research 
DePaul University 
1 East Jackson, Suite 1501 
Chicago, IL 60604-2216 
Phone: 312/362-8403 
Fax: 312/362-5918 
gmclaugh@depaul.edu 



Associate Editor: 

Ms. Debbie Dailey 
Associate Director of Planning and 
Institutional Research 
Georgetown University 
303 Maguire Hall, 37th & 0 St NW 
Washington, DC 20057 
Phone: 202/687-7717 
Fax: 202/687-3935 
daileyd@georgetown.edu 

AIR Professional File Editorial Board 



Managing Editor: 

Dr. Terrence R. Russell 
Executive Director 

Association for Institutional Research 
1435 E. Piedmont Drive 
Suite 211 

Tallahassee, FL 32308 
Phone: 850/385-4155 
Fax: 850/385-5180 
air@airweb2.org 



Dr. T rudy H. Bers 
Senior Director of 

Research, Curriculum 
and Planning 

Oakton Community College 
Des Plaines, IL 

Ms. Rebecca H. Brodigan 
Director of 

Institutional Research and Analysis 
Middlebury College 
Middlebury, VT 

Dr. Harriott D. Calhoun 
Director of 

Institutional Research 
Jefferson State Community College 
Birmingham, AL 

Dr. Stephen L. Chambers 
Director of Institutional Research 
and Assessment 
Coconino Community College 
Flagstaff, AZ 



Dr. Anne Marie Delaney 
Director of 

Institutional Research 
Babson College 
Babson Park, MA 

Dr. Gerald H. Gaither 
Director of 

Institutional Research 
Prairie View A&M University 
Prairie View, TX 

Dr. Philip Garcia 
Director of 
Analytical Studies 

California State University-Long Beach 
Long Beach, CA 

Dr. David Jamieson-Drake 
Director of 

Institutional Research 
Duke University 
Durham, NC 



Dr. Anne Machung 
Principal Policy Analyst 
University of California 
Oakland, CA 

Dr. Marie Richman 
Assistant Director of 
Analytical Studies 
University of California-lrvine 
Irvine, CA 

Dr. Jeffrey A. Seybert 
Director of 

Institutional Research 
Johnson County Community College 
Overland Park, KS 

Dr. Bruce Szelest 
Associate Director of 
Institutional Research 
SUNY-Albany 
Albany, NY 



Authors interested in having their manuscripts considered for the Professional File are encouraged to send 
four copies of each manuscript to the editor, Dr. Gerald McLaughlin. Manuscripts are accepted any time of 
the year as long as they are not under consideration at another journal or similar publication. The suggested 
maximum length of a manuscript is 5,000 words (approximately 20 double-spaced pages), including tables, 
charts and references. Please follow the style guidelines of the Publications Manual of the American 
Psychological Association, 4th Edition. 



© 2006 Association for Institutional Research 







